Machine learning and lean six sigma for targeted patient-specific quality assurance of volumetric modulated arc therapy plans

Highlights • A Machine Learning model highlighted plans with expected low gamma passing rate.• Complexity and expected gamma were monitored prospectively with Lean Six Sigma.• A Poka Yoke system automatically identified plans at risk of failure each day.• Plans considered at risk underwent measurement and were re-optimized if necessary.• Among 1722 volumetric modulated plans, 9 out of 29 at risk were actual failures.

The measurement revealed a GPR of nearly 100%.The first factor which drove the model decision to a low GPR was the small field aperture (FX=30 mm) due to the small planning target volume (3.84 cm 3 ).The arrows in the plots represent the positive (red) or negative (blue) contribution of each feature to the final predicted value of GPR.We hypothesized two concurring factors that could explain this discrepancy.On the one hand, the small field likely caused a lack of lateral electronic equilibrium, forcing the EPID to operate at the limits of its capabilities and leading to an optimistically high GPR.On the other hand, the calculation grid was smaller than standard treatments, i.e., 1.25 mm vs. 2.5 mm.Thus, the Portal Dosimetry algorithm utilized a finer sampling of the fluence, providing a more accurate calculation of the GPR.After a thorough evaluation, the case was considered clinically acceptable.Abbreviations: EPID = electronic portal imaging device; GPR = gamma passing rate; ML = machine learning; PSQA = patient-specific quality assurance; SBRT = stereotactic body radiation therapy.

Figure S2 :
Figure S2: Output of the DSS tool presented to the user for the plan currently opened in the TPS.The table contains ten complexity metrics and expected PSQA outcome for each arc.As visual management, outlier complexities are flagged according to the historical distributions of the treatment site.

Figure S3 :
Figure S3: Boxplots of the complexity metrics in the Measure and Control phase, stratified by treatment.Treatments with more than 40 arcs in the Control phase are reported.The crosses represent the outliers of the distributions (i.e., <5th or >95th percentile).Arrows indicate the direction of increase in complexity for each metric.Abbreviations: Bone mets = bone metastases; Brain & SC = brain & spinal cord; GU = genitourinary; H&N = head & neck; SBRT = stereotactic body radiation therapy.

Figure S4 :
Figure S4: Boxplots of the complexity metrics for different iterations of optimization, stratified by treatment.Treatments with more than 40 arcs in the Control phase are reported.The crosses represent the outliers of the distributions (i.e., <5th or >95th percentile).Arrows indicate the direction of increase in complexity for each metric.Abbreviations: Bone mets = metastases; Brain & SC = brain & spinal cord; GU = genitourinary; H&N = head & neck; SBRT = stereotactic body radiation therapy.

Figure S5 :
Figure S5: Expected GPR before and after re-optimization, stratified by treatment site.Treatments with more than 40 arcs in the Control phase are reported.Significant differences (p < 0.05) according to the Mann-Whitney test are denoted with *.Abbreviations: Bone mets = bone metastases; Brain & SC = brain & spinal cord; GU = genitourinary; H&N = head & neck; SBRT = stereotactic body radiation therapy.

Figure S6 :
Figure S6:Automatic report sent by email in case a plan is considered at risk of PSQA failure and needs further attention from the planner.The report contains the plan information, complexity metrics, and expected PSQA outcome.For explainability purposes, the report also contains a visual representation showing the impact that each feature (complexity metrics and plan parameters) had on the prediction of the ML model.For instance, the plot on the left shows that, starting from a baseline value of 95.1% (light gray at the bottom-right corner) the feature value FX=42.2 mm contributes "-1.03%" at lowering the predicted GPR.The features are ranked from top to bottom by contribution importance.The sum of all the contributions of each feature value brings the model prediction to 87.4%.The baseline value is inferred by SHAP by inspecting the model's tree structure.Abbreviations: GPR = gamma passing rate; ML = machine learning; PSQA = patient-specific quality assurance.

Figure S7 :
Figure S7: Extreme case of abdomen SBRT with 8 and 9 outlier complexity metrics (low complexity) and expected PSQA failure.The measurement revealed a GPR of nearly 100%.The first factor which drove the model decision to a low GPR was the small field aperture (FX=30 mm) due to the small planning target volume (3.84 cm 3 ).The arrows in the plots represent the positive (red) or negative (blue) contribution of each feature to the final predicted value of GPR.We hypothesized two concurring factors that could explain this discrepancy.On the one hand, the small field likely caused a lack of lateral electronic equilibrium, forcing the EPID to operate at the limits of its capabilities and leading to an optimistically high GPR.On the other hand, the calculation grid was smaller than standard treatments, i.e., 1.25 mm vs. 2.5 mm.Thus, the Portal Dosimetry algorithm utilized a finer sampling of the fluence, providing a more accurate calculation of the GPR.After a thorough evaluation, the case was considered clinically acceptable.Abbreviations: EPID = electronic portal imaging device; GPR = gamma passing rate; ML = machine learning; PSQA = patient-specific quality assurance; SBRT = stereotactic body radiation therapy.

Figure S8 :
Figure S8: The PSQA program adopted in our department.After the Lean Six Sigma implementation, an extra layer of control for monitoring complexity and expected PSQA was introduced.Abbreviations: PD = Portal Dosimetry; PSQA = patient-specific quality assurance.